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As Rhode Island Commissioner of Education 
team began implementation of Race to the Top, they knew that a crucial ingredient 
for the State’s success would be its system of performance management for the $75 


million award. They needed a process to col 
Rhode Island Department of Education (RI 


Deborah A. Gist and her leadership 


ect and analyze pertinent data from the 


D 


E) and participating local educational 


agencies (LEAs), a transparent structure for discussion and decision-making based 
on the data, and a way to track action items intended to address challenges. 


The State Education Agency (SEA) Capacity 
Building work stream of the Reform Support 
Network (RSN) seeks ways to strengthen the 
organizational capacity of SEAs by supporting 
their ability to implement and sustain their 
proposed Race to the Top reforms over time. 
To accomplish its purpose, the SEA Capacity 
Building work stream seeks to support SEAs in: 
+ Sharpening and defining their roles as 
they move from compliance to support; 
Building system capacity and alignment, 
specifically in the SEA and local educational 
agency (LEA) relationship, ensuring that 
resources such as finances, data systems 
and personnel support reform goals; and 
Delivering on their reform goals over 
time, ensuring that priorities such as 
equitable distribution of teachers, 
development and/or choice of high quality 
instructional materials and closing of the 
student achievement gap are realized. 


Using webinars, case studies, working groups, 
connections with experts and onsite technical 
assistance, the SEA Capacity Building work 
stream focuses on three interconnected areas: 

Performance Management 

Expanded System Capacity 

Sustainability 


outcomes. The structured, ongoing system of 
information gathering and data analysis includes 
RIDE project managers update Commissioner 

Gist and RIDE’s leadership team, answer questions 
to overcome State and local challenges inherent in 


implementing bold reforms. 


RIDE leaders say that EdStat gives them a strong 
handle on how the agency is managing both 
the process and the quality of Race to the Top 
implementation across all LEAs. EdStat is one of 
several performance management tools used 
by RIDE. It helps them answer key questions 
about Race to the Top progress: What did we say 
we would do? Is that happening? How well is it 
going? What's getting in the way? How do we 
know? RIDE leaders also credit the introduction 
of EdStat in June 2011 with shifting the agency’s 
culture in noticeable ways. View an interview with 


Commissioner Gist _ p>» 
— 


The Reform Support Network, sponsored by the U.S. Department of Education, supports the Race to the Top 
grantees as they implement reforms in education policy and practice, learn from each other, and build their 
capacity to sustain these reforms, while sharing these promising practices and lessons learned with other 
States attempting to implement similarly bold education reform initiatives. 


Enter “EdStat”—a system of performance 

management that has enabled RIDE to monitor 

the progress of its participating LEAs and its own 

offices on Race to the Top implementation and 

almost-weekly two-hour meetings during which 

about progress or challenges and strategize ways 


“EdStat is something | think we use not only to get 
results and track performance and solve problems,’ 
Commissioner Gist said. “It also serves the role of 
changing the culture and helping the members of 
our team—who are incredibly smart and educated 
and wonderful—to think about performance in a 
different way, to think about tracking our progress 
in ways that are much more embedded in data and 
results.” 


The “Stat” Process 
What is “Stat”? 


The term “stat” refers to a process through which 
agencies track data for a particular project or 
initiative, analyze and share the data with leadership 
teams for discussion, develop a concrete action plan 
with resources and follow up to measure progress. 
City governments and police departments have 
used the “stat” process to pinpoint trends—sudden 
spikes in crime in a police precinct, for example, or 
availability of recycling bins in a neighborhood— 
and devise targeted solutions. One of the key 
features is that the managers of the offices whose 
data are being presented meet directly with senior 
leaders to answer questions, request feedback or 
get support and advice. Ideally, the data and the 
problem under analysis link to one of the agency’s 
larger strategic challenges or goals. 


Bringing “Stat” to a State 
Education Agency 


Commissioner Gist used the “stat” process when 
she served as State superintendent in the District of 
Columbia, and felt that it would be a valuable way 
to monitor the Race to the Top grant progress. She 
brought it to RIDE with the help of UPD Consulting, 
a firm that has helped LEAs and governments 
employ “stat” as a performance-management tool. 
EdStat started as a tool just for Race to the Top 
projects, but RIDE has broadened it to include 

four of the five areas in its five-year strategic plan, 
which was the foundation for its Race to the Top 


Rhode Island Public School 
Demographics (2011-2012) 


300 public schools 


54 LEAs (including 13 independently operated 
charter schools and one regional collaborative) 


142,854 students 
44% of students eligible for subsidized lunch 
6% of students receive English learning services 


16% of students receive special 
education services 


64% Caucasian; 22% Hispanic; 
8% African-American 


Source: http://infoworks.ride.ri.gov/state/ri 


Race to the Top Initial 
Award Details 


Award amount: $75,000,000 
50 participating LEAs, 2 involved LEAs 
298 participating or involved schools 


141,311 students in the participating 
or involved schools 


Source: Rhode Island Department of Education 


application. In using EdStat to manage change 
across the agency, RIDE hopes to bring overall 
coherence to its planning efforts. View an interview 
with Mary-Beth Fafard and Alaina Restivo ( 


— 
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EdStat at RIDE generally follows this sequence: 


RIDE decides which projects to elevate to an 
EdStat meeting. Forty-one projects, the majority 
of which are Race to the Top projects, are 
grouped together for regular EdStat reviews 
based on the strategies to which they are 
aligned in the department's five-year strategic 
plan. 


RIDE conducts an EdStat review of each project 
grouping every four to six weeks, roughly one 
EdStat meeting each week. Within each EdStat 
meeting, there are topics, and each topic is an 
overview of the major bodies of work or projects 
under that area. 


Before the meeting, project teams update their 
action items from the previous meeting and 
provide data relevant to the issues that the 
current EdStat meeting will discuss. Each project 
team forwards their data and action items 

to the performance management executive, 
who meets with the project team and writes a 
briefing memo analyzing the data according 

to a template, including relevant work plan 
updates and context for the data. 


The performance management executive and 
the project team review the briefing memo 
together and discuss strategies to address the 
data and solutions as needed. 


The performance management executive 
prepares an annotated version of the briefing 
memo, highlighting suggested areas for 
discussion, and sends this to the EdStat panel 
—composed of Commissioner Gist, her division 
chiefs and the State’s Race to the Top strategic 
coordinator—who review it in advance of the 
meeting. 


The commissioner facilitates the EdStat 
meetings, which typically last 90 minutes. 
Additionally, there is a 15-minute “prebrief” 
of the panel to review the memo content, 
and a 15-minute debrief afterward, when the 
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performance management executive discusses 
the meeting's accomplishments with the panel. 
The project lead and division chief in charge, as 
well as the project team, walk the panel through 
the key projects and data to be discussed, as 
well as updates from the previous meeting. 


EdStat Panel Leadership Team 


Deborah A. Gist, Commissioner 


David V. Abbott, Deputy Commissioner and 
Chief Legal Officer 


Andrea Castaneda, Chief of Accelerating 
School Performance 


Carolyn Dias, Chief of Fiscal Integrity and 
Efficiencies 


Mary-Beth Fafard, Race to the Top Strategic 
Coordinator 


Clark Greene, Chief of Staff 


Mary Ann Snider, Chief of Educator 
Excellence and Instructional Excellence 


Panel members ask questions about the data, 
trends and unanticipated findings, and project 
leads bring up issues for which they need the 
panel members’ support. 


The performance management executive keeps 
a running list of action items to be reviewed 

by the attendees before the meeting ends. 
Action items can include follow-up phone calls 
with SEA or LEA staff, requests for more data 
gathering or strategic communications by the 
commissioner. 


The performance management executive 
debriefs the meeting with the panel and 
provides feedback and overall thoughts. 


The project team and the panel begin or 
complete their assigned action items prior to 
the next EdStat meeting. 


RIDE staff on both sides of the EdStat table—the 
project teams and the panel—mostly praise the 
discipline and focus that EdStat has brought. But 

it was not an easy transition, staff members recall. 
When UPD held its first EdStat training in April 2011, 
“There was enormous hesitation at every level of the 
agency,’ recalled Mary-Beth Fafard, RIDE’s Race to 
the Top strategic coordinator. 


“The first thing—no matter what the commissioner 
said—people feared it was to get you: ‘We're going 
to figure out which staff are not performing, and 
we are going to get rid of them and hold them 
accountable for not being able to do this work even 
when you haven't been given the resources, time or 
support: And that was where the biggest hesitation 


” 


Was. 


To address the hesitation, the commissioner and 
her staff reiterated that EdStat was about support, 
not staff punishment. They also emphasized that 
EdStat was personalized for Rhode Island's needs. 
For example: 


Unlike other agencies that have used the “stat” 
process, EdStat is not about an employee's 
individual performance or his or her personnel 
evaluation—perhaps “the hardest thing to 

get across to people,’ said Alaina Restivo, the 
performance management executive at RIDE. 


Meetings will not dwell on an employee's 
negative performance—even if it shows up in 
the data—because that person's manager will 
handle the issue. 


The performance management executive drafts 
the memo, and the project teams contribute 
and verify the accuracy of the memo content; 
however, they do not write the memo, and the 
analysis of progress towards goals is completed 
by the performance management executive for 
the purpose of creating an objective analysis. 


The memo is one of the unique features of 
EdStat, and panel members set aside time to 


read it in advance so they can be prepared for 
the detailed discussion at EdStat meetings. 
The document helps focus EdStat meetings by 
providing, in a clear and consistent format, the 
project goals, measures of success, relevant 
data, implementation activities and risks/ 
challenges. In addition to providing information 
to the panel, it provides a framework for 
discussion and keeps panel members on task. 
For a sample annotated memo, see Appendix 
A: Race to the Top Performance Management 
Memo: Educator Effectiveness. 


- Not every project is “statted” The 41 projects cut 
across four different assurance areas that also 
align to RIDE's strategic plan. RIDE found that 
challenges in one project also affect others, so 
RIDE bundled projects for efficiency to avoid 
multiple and overlapping EdStat sessions, 
with the added benefit of breaking down silos 
between the offices that handled these projects. 


One challenge was tracking the early stages of Race 
to the Top projects, when milestones tended to 

be more process-oriented, such as initial trainings 
with LEAs. The attention to limited measures at first 
seemed incongruent with EdStat's purpose in the 
minds of RIDE project team leads who had crafted 
long-term student achievement goals. But EdStat’s 
focus on the milestones critical to successful 


“Performance and conversations 
about results and tracking progress 
and solving problems—all of that 

is permeating throughout the 
agency in the way we work with 
one another. I do think EdStat 
played a role to anchor and serve 
as a model of how to do that." 


Deborah Gist 
Commissioner of Education, 
Rhode Island Department of Education 
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implementation helped get them off the ground. 
View an interview with Mary-Beth Fafard and Alaina 
Restivo p> 

— 
“Initially, it was really hard for us. When you're in 
that mode of designing brand-new systems and 
protocols, you don’t have a whole lot of data, and 
sometimes the question of quality is more about, 
‘Are you just keeping up with the pace? Are you 
getting things delivered? Are people coming to 
trainings?” said Mary Ann Snider, chief of educator 
excellence and instructional effectiveness at RIDE. 
“We began by focusing on things that were easy 
to measure and easy to talk about. So the team’s 
frustration was that there were 50,000 other things 
behind that one thing that we were focusing on 
in the EdStat process, which we were also deeply 
concerned about and knew ultimately would have a 
large impact on the overall quality of the work. But 
we were reducing the content to something very 
narrowly focused. In retrospect, we needed to do 
that to get where we are today.’ 


A Typical EdStat Meeting 


A senior consultant from the Reform Support 
Network observed the EdStat meeting that took 
place on June 6, 2012. EdStat meetings have a 
uniform agenda, starting with a “pre-brief” between 
the facilitator and the panel. The panel and the 
project team then discuss each project's action 
items, measures of success, data on assessment 

of implementation (with red, yellow and green 
indicators), and the risks and challenges set out in 
the briefing memo. Sometimes, projects have an 
appendix of supporting data to assess the quality of 
implementation. 


Commissioner Gist runs the meetings, and the 
performance management executive tracks action 
items on her computer, whose screen is projected 
onto the wall for all to view. 


At the EdStat meeting on June 6, RIDE focused on 
the project work supporting the agency’s educator 
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effectiveness goals as defined in RIDE's strategic 
plan: 


- By 2012, 75 percent of educators will be 
evaluated using a rigorous performance 
evaluation that includes student achievement 
data. 


By 2015, all students will be supported 
by educators who demonstrate effective 
performance. 


The three topics up for discussion under these 
goals were new teacher induction, certification 

and recruitment, and educator evaluation. 
Commissioner Gist’s leadership team sat on one 
side of the table, and the project team sat across 
from them. The bulk of the meeting was devoted to 
discussion of each project’s successes, challenges 
and next steps. For the last 15 minutes, the 
project team was excused, and the performance 
management executive debriefed the panel on the 
session's accomplishments and areas to watch for 
the next meeting. 


The meeting was efficient, with no sidebar 
conversations. Ninety minutes passed before 
any participant checked his or her cell phone. 
Questioning was focused but not aggressive or 
pointed, and any data analysis that highlighted a 
challenge resulted in an action item. When survey 
data suggested that one LEA was struggling with 
a Race to the Top project on induction coaches for 
beginning teachers, Commissioner Gist asked that 
a member of the project team follow up with her 
point of contact in the LEA—with the expectation 
that the project team would provide an update at 
the next EdStat meeting. Participants were similarly 
focused on numbers when they explored the 
number of alternatively certified teachers. View a 
video clip of an EdStat Meeting (pm 

— 


The panel pushed the team to be concrete. 
Regarding the caseload of induction coaches, 
Commissioner Gist asked, “It says in the memo that 


you're anticipating a larger than usual number of 
retirees. Is that anecdotal?” 


The answer was “yes.” The project team expected 
more retirees, which would lead to more vacancies, 
which potentially would lead to larger caseloads 
of beginning teachers for induction coaches. Since 
coaches are expected to provide an average of 75- 
90 minutes of support a week for each beginning 
teacher, knowing their expected caseload is critical 
to managing their time. But projected estimates of 
retirees thus far have been “chatter not grounded 
in actual data, explained Lisa Foehr, the director 

of educator quality and certification, who leads 
the team. Without concrete data from LEAs about 
anticipated retirements, the project team would 
have difficulty accurately forecasting caseload and 
deployment of induction coaches. 


After discussion, Commissioner Gist said, “Rather 
than these guessing games around data sharing, 
around better reporting, do we want to capture 
that, maybe not as a short-term action item, but 
something we want to keep in the parking lot? We 
could set up a meeting with the chairman of the 
retirement board.’ The suggestion to move forward 
on a data-sharing agreement became an action 
item for the deputy commissioner and chief of fiscal 
integrity and efficiencies. 


Sometimes, it is more difficult to draw conclusions 
from EdStat data. At the same meeting, the group 
discussed the State’s new software program for 
posting vacancies, called “School Spring."The 
vendor report showed that 34 percent (or 18) of 
the LEAs participating in that project reported not 
using School Spring. This was contrasted with the 
LEA quarterly progress self-assessment, where 72 
percent reported that they were on-track. Foehr 
cautioned that the data report from the vendor was 
reporting on the postings on a single day, rather 
than over time, and they had since requested more 
detailed reports from the vendor, which might 
reveal LEAs using School Spring at different times 
during the year. The panel requested an updated 
analysis be included in the next EdStat meeting. 


The meeting ended with Restivo, the performance 
management executive, recapping the action items 
and then debriefing the panel after Commissioner 
Gist thanked the project team for its time. Restivo 
coached the panel on what to do differently the 
next time. “You did a good job covering the robust 
content under induction,’ she said. “I thought you 
guys worked really well together.” But she also told 
them to be more disciplined in their questioning 
the next time they received an update on recruiting, 
and to distinguish between curiosity about a project 
versus pushing for performance. View a video clip of 
an EdStat Meeting (p 

— 


One of the advantages of EdStat, according to RIDE 
staff, is that the agency can respond to problems 
and propose solutions before the problems escalate. 
Project teams benefit from having RIDE’s top staff 
observing and advising in response to trends— 
particularly in response to data coming from LEAs, 
who gather quarterly for “Collaborative Learning 

for Outcomes” meetings, or CLOs (see additional 
publication, “Collaborative Learning for Outcomes: 
Connecting LEAs with the Rhode Island Department 
of Education’). The CLO process captures LEA 

data that subsequently feed into EdStat meetings, 
connecting RIDE to its LEAs in a way that did 

not exist before, and providing a window into 
challenges LEAs are facing. For a sample CLO 
progress report, see Appendix B: Race to the Top 
Collaborative Learning for Outcomes (CLO) Group 

1 — May 15, 2012. 


For example, one of RIDE's Race to the Top 
deliverables was a “study of the standards” project 
that involved training 4,300 educators in the 
Common Core State Standards (CCSS) by August 
2012. On paper, it appeared that schools were 
meeting their training attendance requirements. But 
an EdStat review of attendance data disaggregated 
by educator role made it clear that schools were 

not necessarily sending the core instructional staff 
to whom the training was geared. That discovery 
enabled RIDE to clarify attendance guidelines and 
make sure to sponsor enough trainings for LEAs that 
were not sending the correct staff or encourage 
them to send core staff. It also led to a discovery 
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that some collective-bargaining agreements did 
not allow the schools to define who could attend. 
The team set a goal for the percentage of training 
attendees who were considered “high” priority for 
Common Core implementation (for example, high 
school English teachers) versus “low” (for example, 
school nurses). 


EdStat also enables cross-functional decisions across 
RIDE offices that previously would have required 
more meetings, memos or emails. A question 
concerning the alignment of the State's virtual 
math modules with the CCSS led to the project 
team’s asking one of the State's partners, the Dana 
Center, to undertake a review. The review led to 
clarifications as to whether the contract with the 
Dana Center should be amended—a small detail 
that saved time because an omission was caught 
early. 


“Pre-EdStat, we'd all have said we're committed to 
excellence, we want to have good products, and all 
trainings and all the things we are doing would be 
of the highest quality. But they were rather vaguely 
stated, and if things started to drift away from that 
initial vision, there really wasn’t a way to catch it in 
process as it was unraveling,’ Snider said. “It would 
be a year later that we'd say, ‘Well, that didn’t work 
out so well’ but we didn’t have any opportunity to 
call that out as it started to unravel. 


Advice for States 


Commissioner Gist has employed EdStat at 

two different SEAs and believes that any SEA, 
regardless of its size, could avail itself of the process 
successfully. She and other RIDE staff involved in 
EdStat offer advice to States interested in creating a 
similar performance management system: 


* Engage the support of leadership: The SEA’s 
leader, usually a commissioner of education 
or State superintendent, must believe in the 
process and reinforce the value with staff. 


¢ Perserverence: The leadership team also must 
also perservere through the growing pains 
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“Pre-EdStat, we'd all have said 

we te committed torexcellenee we 
want to have good products, and 

all trainings and all the things we 
are doing would be of the highest 
quality. But they were rather vaguely 
stated, and if things started to drift 
away from that initial vision, there 
really wasnt a way to catch it in 
process as it was unraveling.” 


Mary Ann Snider 
Chief of educator excellence 
and instructional effectiveness at RIDE 


of training and initial execution and reinforce 
with tenacity that the time commitment is 
worthwhile. 


Customize the process: SEA staff cannot 
learn the EdStat process through a book or 

a PowerPoint; it must be customized to each 
agency's needs and vision. RIDE, for example, 
allowed project teams to offer feedback to 
shape the EdStat memos that precede each 
meeting. Such participation does not occur 
in all stat processes, but fits the culture that 
Commissioner Gist wanted to build. 


Learn by doing: RIDE has adjusted EdStat's 
format and structure (bundling related projects, 
for example), especially during the start- 
up phase, with the purpose of continuous 
improvement to ensure the meetings fit RIDE’s 
needs. SEAs should not be afraid to fine-tune 
their performance management systems; no 
process should be static. 


Assign staff to work on EdStat: With an EdStat 
meeting nearly every week, the performance 
management executive's role on EdStat 
amounts to a full-time assignment—compiling 
the data, meeting with project teams, writing 
the memo, facilitating the EdStat sessions and 
following up on action items. Depending on 


their size, SEAs may need at least one person 
dedicated to EdStat management. Additionally, 
depending on the culture of the agency, a 
background in organizational dynamics and 
change management would be helpful. 


Make the time commitment: Both project 
teams and the panel must abide by the 

time commitments of EdStat, which include 
preparation and follow-up post-meetings in 
addition to the two-hour stat meeting. Also, 
the EdStat panel identifies action items during 
the course of the meeting for both project 
teams and panel members, and they expect the 
assigned staff member to complete those items 
within the anticipated timeline. At RIDE, EdStat 
time is sacred because the return on investment 
of that time is so evident. 


Ensure participants understand their roles: 
The role of the panel is not to make insinuations 
or inferences about project personnel or to 
micro-manage the project team’s work. It is 
rather to push the team to think broader and 
deeper, and to provide perspective, expertise 
and assistance that will help the team succeed. 
The role of the project team is to provide 
context and candor about successes and 
challenges, not to sugar-coat or to engage in 
blame. The performance management executive 
should maintain an objective viewpoint, helping 
the project team understand the kinds of 
questions the panel may ask, while coaching the 
panel on engaging thoughtfully with project 
team members. 


Commissioner Gist said that EdStat likely will 
continue after Race to the Top. “Performance and 
conversations about results and tracking progress 
and solving problems—all of that is permeating 
throughout the agency in the way we work with 
one another, she said.“! do think EdStat played a 
role to anchor and serve as a model of how to do 
that’View an interview with Commissioner Gist (pS 


— 


Questions to consider: 


1. What are the most interesting features of EdStat, 
in your mind? 


2. Cana performance management system really 
change the culture of an SEA? 


3. What other building blocks have to be in place? 


4. Do numbers tell the whole story with project 
implementation? What other considerations 
should EdStat take into account? 


5. Can EdStat be adopted for your agency? Why or 
why not? 


See Appendix C: The EdStat Process in Action, for 
an illustration of how a problem gets discussed at 
an EdStat meeting and what data points inform the 
discussion. 


This publication features information from public and private organizations and links 
toadditional information created by those organizations. Inclusion of this information 
does not constitute an endorsement by the U.S. Department of Education of any 
products or services offered or views expressed, nor does the Department of 
Education control its accuracy, relevance, timeliness or completeness. 
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Race to the Top Performance Management Memo: 
Educator Effectiveness 
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edstat Race to the Top Performance Management 


ede a ee ae de Memo: Educator Effectiveness 


RHODE ISLAND 
—____—_ - Last EdStat: Today’s Date: Next EdStat: 


Department of Education 6/7/2012 7/13/2012 8/23/2012 


Ensure Educator Effectiveness 
All educators will be effective and committed to accelerating student performance. 


Project Sponsor and Mary Ann Snider, Chief of Educator Excellence and Instructional Effectiveness 
Managers Lisa Foehr, Director of Educator Quality and Certification 

Tim Heavey, Educator Quality Specialist 

Sarah Whiting, Educator Quality Specialist 

Jessica DelForge, Educator Quality Specialist 

Andre Audette, Educator Quality Specialist 

Mary Keenan, Educator Quality Specialist 

Hilda Potrzeba, Educator Quality Specialist (tentative attendee) 

Donna Okrasinski, Educator Quality Specialist (tentative attendee) 


AGENDA 
a TOPIC PANEL LEAD START/END TIME TIME 
1 New Teacher Induction Deborah & Clark 12:45 AM—- 1:00 PM 15 MIN 
SY12-13 Planning 
2 Certification & Recruitment Deborah & Mary-Beth 1:00 PM — 1:15 AM 15 MIN 


Alternative Certification Pathways (TNTP and TFA) 
Certification Redesign 
Certification Systems (RICERT & Current System) 


3 Educator Evaluation Deborah & Dave 1:15 — 2:00 PM 45 MIN 
SY11-12 LEA Summative Rating Reporting 
SY12-13 Planning 

EPSS Implementation and Build 

Wrap up Alaina 2:00 - 2:15 PM 15 MIN 


OVERALL NOTES: 
In addition to out-of-state guests, there are two staff members attending who are new to EdStat. Jessica DelForge is the 


new Educator Quality Specialist focused on the summer evaluator training. Tamika Pollins is attending as the project 
lead for Student Learning Objectives. 


e TOPIC 1 (INDUCTION) My suggestion is to focus on the data collection processes used by the team, and how 
they can strengthen the collection in order to assess progress towards goals in the upcoming year. Additionally, 
we should make sure the timing/plan for the ‘parking lot’ action items is understood by all. 


e TOPIC 2 (CERT & RECRUITMENT) Both alternative certification vendors have missed their cohort size goal of 30. 
We should determine what implications, if any, this has on RTT and if additional actions need to be taken. 


e TOPIC 3 (EVALUATION) There is an action item around discussing Student Learning Objectives. Additionally, we 
have started receiving data from the first evaluator training academy. From reviewing the data, evaluator 
confidence and understanding of SLOs is still not as strong as the other evaluation components. | would use that 
data to frame the discussion of SLO training and support in the upcoming year. Lastly, topic 3 contains 
preliminary summative ratings from 5 LEAs. It would be helpful to understand what level of discussion the panel 
would like to have about this data moving forward. 
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TOPIC 1: New Teacher Induction 
12:45-1:00PM_ | Review the data from the end of year survey and discuss ways that the team is using that feedback to 


(15 MIN) adjust their approach for the upcoming year. 
|. ACTION ITEMS: 
# ITEM WHO TIMING STATUS 
1 Follow-up with [LEA 1] regarding their CLO Challenge/feedback Hilda July 12th Complete 
2 Monitor confirmed hiring of beginning teachers over SY11-12 numbers Team Ongoing [see memo] 
3 | Meet w. Retirement Board to discuss data sharing agreement with RIDE CD/LF/MAS Parking Lot | N/A 
4 | Initial draft of barriers to more effective/efficient human capital systems MAS/LF/DVA | Parking Lot | N/A 
5 | ALT discussion re: statues & policies to support human capital planning MAS w. LF Parking Lot | N/A 


- Item #1: [LEA 1]s feedback was that the central office knew little about IC support/assignments. What did Hilda learn? 
- Item #2: cover in questions below 
- Remaining items. What does ‘parking lot’ mean? Next quarter versus next year? 


Il. MEASURES OF SUCCESS: 
1. 75% of beginning teachers agree/strongly agree that the Induction Coach model is valuable (July 2012) 
2. 75% of building administrators will agree/strongly agree that the Induction Coach model is valuable (July 2012) 
3. 100% of participating districts will confirm the number of beginning teachers needing Induction Coach support (Sept 2012) 
4. By Sept, the team will recruit/retain coaches as to maintain a caseload of < 17 BTs per IC (Starting Sept 2012) 


See additional data-related questions in Appendix 1. 

- | Measure #3: How many RTT LEAs are ‘participating’ AND have vacancies that might be filled by a beginning teacher? This 
should inform the goal AND planning. i.e. if there are 70 vacancies then there can’t be more than 70 beginning teachers. 

- OTHER: Is the team planning to streamline minutes of support per week reported during SY12-13? 


Ill. ASSESSMENT OF IMPLEMENTATION: 


Task Completion RIDE’s Quality of LEA Participation 
Implementation 


SY12-13 New Teacher Induction [Hilda & Donna] 


- | ASSESSMENT -— Quality: The CLO data, shared in earlier EdStat sessions, suggested that some district and building 
administrators feel that they had insufficient communication about beginning teacher support during the year. 
O How does that factor into the quality assessment? 
O What goals or aspirations does the team have around improving this communication in the upcoming year? 
O Howwill they collect and report on that data during SY12-13? 
- ASSESSMENT - LEAs: What evidence is being used for this assessment? Can this be based off Measure #3 in SY12-13? 


The SY12-13 Induction program is on track. The team has confirmed LEA participation for next year and, for those 
participating, is starting to receive estimates regarding the number of beginning teachers. They anticipate that caseloads will 
be initially drafted by the end of July or early August (much earlier than last year), but will likely not be confirmed until 
September. See above questions around Measure of Success #3. 


Since the last EdStat, 6 candidates were offered induction coach positions for the 2012-2013 school year. In addition, nine 
candidates were offered alternate spots and the opportunity to receive training over the summer. Three LEA-sponsored 
educators will receive the full training as non-release coaches to support the district as they feel appropriate. During June, the 
team met with LEA HR staff to reinforce quality standards for collecting and reporting beginning teacher data. 


Upcoming milestones include: 
- Finalize LEA participation (April — July 2012) - COMPLETE 
- Recruit and select new coaches (May 2012) — COMPLETE 
- Train new induction coaches (June-July 2012) — On Track 
- Schedule 2012-2013 academies and forums (June 2012) - COMPLETE 
- Match coaches to new teachers (June-September 2012) — On Track 


IV. RISKS AND CHALLENGES: 
None. 
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APPENDIX 1: Induction Program End of Year Survey Results 


Beginning Teacher Support Timing 
| began working with my Induction Coach... 


5.2% 


m™ Before | began Teaching 

My first week of school with students 
m Weeks after starting my position 

= Months after starting my position 


The data above indicates that 5% of beginning teachers began working with their coach months after starting. 
- Has the team analyzed this data compared to when they confirmed the hire and matched the BT to a coach? 
- Does the team have goals related to this data for the upcoming school year? 


Induction Coach Efficacy 
Percent Agreement 


100 


Percent Agreeing 


My induction coach has helped me My work with beginning teacher(s) Induction coaches are increasing 
to be an effective teacher improves their teaching practice beginning teacher effectiveness 


The data above is related to Measure #1 & 2: 
The data presented suggests that 95% of Beginning Teachers and 97% of Building Admin agree that the induction program increased 
beginning teacher effectiveness. 
-  Isthe ‘effectiveness’ a yes/no question? 
- Will that data, and other data collected, make a strong case for the value of the Induction Coach model more than other 
district-provided resources? i.e. sustainability 
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TOPIC 2: Certification Redesign & Alternative Certification Pathways 
1:00 — 1:15 PM Review progress towards cohort placement for TNTP and TFA 
(15 MIN) 


I. ACTION ITEMS: 


Il. MEASURES OF SUCCESS: 
1. By SY12-13, a redesigned licensure renewal process will be fully implemented (on track) 
2. 100% of participating LEAs with openings utilize SchoolSpring during the SY12-13 hiring season (on track) 
3. By September 2013, 100% of initial certification candidates will take the new regents-approved licensure tests 


Ill. ASSESSMENT OF IMPLEMENTATION: 


Task Completion RIDE’s Quality of LEA 
Implementation Implementation 


Alternative Certification [Hilda] 
Certification Redesign [Andre and Mary] 
Certification System [Shoba] 


Alternative Certification (TNTP and TFA): 

Both TFA and TNTP have reported final cohort sizes that are slightly below the goal of 30 corps members/fellows. Since the last 
EdStat, TNTP completed interviews and has a final cohort of 28 fellows; as of July 10", they have placed 4 out 28. TFA has 
placed 23 out of 29 corps members. 


NA 


NA 


- | What are the RTT implications for not meeting the targeted TFA/TNTP cohort size? 
- | How confident is TNTP that they will be able to place the remaining 24 fellows? 


Certification Redesign: 

Since the last EdStat, the Board of Regents approved and passed 45 of 46 proposed certification tests. The Board of Regents 
requested that the English Speakers for Other Languages (ESOL) certification test be further studied and additional information 
brought before the Regents for their consideration. They have reviewed this additional information and are scheduled to vote to 
on the ESOL test on July 19'". The team does not anticipate this will delay the September 2013 implementation of the licensure 
tests; in parallel, they have begun developing communication materials for the 45 regents-approved tests and passing scores. 


Upcoming milestones include: 
- Process for review/collection of professional development per certification regulations (Fall 2012) 
- Process for reinstatement of a certificate lost due to poor job performance (Fall 2012) 
- Policy work around certification redesign (ongoing) 


Certification System: 
The timelines for the certification system (now called eCert) have been pushed back, and the system is now planned for a 


November release. The business requirements have been reviewed and finalized, and the project is now on track based on the 
new timeline. 


Upcoming milestones include: 
- Completion of business rules for RICERT (August/September) — On Track 
- Testing of new RICERT (June - October 2012) — On track 
- Launch of Personnel Data Collection RICERT add-on (Fall 2012) — On Track 


IV. RISKS AND CHALLENGES: 
e = If not approved on July ‘io. approval of the ESOL test may present challenges to implement that specific certification 
test on schedule (i.e. with the recently approved tests that will go into effect on 9/1/13). 
e =Although the eCert timeline will not substantially impact users, it does have cost implications because it requires the 
team to maintain the existing system. 
e The team has not yet been able to procure Kevin’s expertise on a consultative basis. While the contract is in the works, 
they need to clarify whether prior state employees can serve as consultants. 


- Does the team need help determining how to cover the Kyran costs, or is that already settled? 
- | What will happen if the team cannot contract Kevin on a consultative basis? 
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TOPIC 3: Educator Evaluation 
1:15-2:00 PM The team should present an overview of LEA SLO support and training for SY12-13. Additionally, the 


(45 MIN) panel should review the Academy 1 training data and training registration data. 
I. ACTION ITEMS: 
# | ITEM WHO TIMING STATUS 
1 Escalate LEA Building Admin registration/participation as needed Lisa Foehr As needed [Appendix 4] 
2 Determine LEA contract assistance needed re: BEP personnel policy req’s Lisa w. DVA Parking lot | N/A 


- Review school-level data noted in CLO meeting notes and QPU reports 
- Determine the nature/level of detail and timing of sample language 

- Create a map of CBA lifespan across LEAs (i.e. who is 'up' for renewal) 
- consider training with school committees and superintendents 


3 Review LEA EPSS concerns (technical, access by RIDE to LEA data, etc.) MAS, LF w. MF Ongoing In Process 
4 | Communicate aspects of evaluation completed in the EPSS to the field Team Ongoing In Process 

5 | Add LF/MAS to PVD biweekly meeting agenda in order to discuss EPSS Andrea C. ASAP Not Needed 
6 | Review RIDE access to data (absenteeism, etc) as Q1 deliverable MAS @ DGB Q1 (Sept+) | N/A 


- Item #1: See Appendix 4 for a list of LEAs who have not yet registered. What outreach has the team done already? 

- Item #2: This is a long term item and the panel suggested several interim steps. Our notes also indicated that the team 
would present differentiated strategies at the next EdStat. Given all that is happening on the Eval side, that was likely too 
aggressive — however, the panel should review what progress has been made since the last EdStat. 

- Item #3-5: | suggest discussing these as part of the EPSS topic below. 


Il. MEASURES OF SUCCESS: 
1. 100% of districts submit aggregate final evaluation summative ratings (July 2012) [SEE APPENDIX 3] 
2. 100% of districts register and participate in summer academies/applicable training [SEE APPENDIX 3] 
3. 80% of training attendees agree that the session met its intended objectives [SEE APPENDIX 4] 
4. 100% of districts have a District Evaluation Committee — IN PROGRESS (90% of LEAs submitted to RIDE) 


For Measures #1, #2 and #3, please see questions embedded in Appendix 3 and 4. 
- | Measure #3: What districts have not provided District Evaluation Committee names to RIDE? What outreach was done? 


Ill. ASSESSMENT OF IMPLEMENTATION: 


Task Completion RIDE’s Quality of RIDE’s Assessment of 
Implementation LEA Participation 
SY11-12 Evaluation Implementation NA NA 


SY12-13 Evaluation Planning & Implementation 
- Summer Academy Training 

- FFTPS Calibration Tool 

- Messaging/Support for SLOs 

Educator Performance Support System 


SY11-12 Implementation 
Since the last EdStat, the team has started collecting SY11-12 summative ratings from LEAs [SEE APPENDIX 4]. 


SY12-13 Implementation: 

The team is on-track for implementation in the upcoming school year. They have hired an evaluation specialist and a district 
fellow (a former ISP from [LEA 2]). They approved the evaluation model proposal for The Learning Community. Additionally, 
since the last EdStat, the team has distributed building admin and teacher guidebooks to all districts. Based on the summer 
training pre-survey data, the majority of evaluators received and reviewed the guidebook. 


Summer Training Academies [Sarah Whiting] 

There are five 4-day Academy trainings being offered from June through August and all teacher and building 
administrator evaluators must attend one of the Academy training sessions. All facilitators have been trained and the first 
Academy (week 1) is complete. The team has done some initial analysis on the survey feedback [SEE APPENDIX 3] but will 
continue to review the feedback from the first two Academies in order to inform content changes in time for the final 
Academy trainings (weeks 3-5). 


School Year Implementation and Planning [Lisa Foehr] 
The team is comparing the number of ISP support days committed to LEAs and comparing that caseload to the ISPs 
already hired. Based on that analysis, they will determine whether they need to utilize Summer Academy facilitators as 
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ISPs. Two have already been contacted to support LEAs moving forward. The ISP hiring and training, along with LEA 
assignments, will be finalized by September. 


“FFTPS” Calibration Tool [Sarah Whiting] 
The team has provided ETS TeachScape with usernames and passwords so that evaluators can log into the FFTPS 
observation calibration tool during the summer academy training. 


See risks and challenges below. 
O_ Based on the survey data and feedback from the sessions, how is the FFTPS enhancing evaluator calibration? 
O The team has an expectation that evaluators will complete 10+ hours of PD over the course of the year using the 
FFTPS tool. How will they provide LEAs with data to track whether this is actually being done? 


Student Learning Objectives [Lisa Foehr and Tamika Pollins] 

The exit survey analysis from Modules 3 and 4 evaluator training indicated that evaluators would like to see better 
guidance on how to choose appropriate SLO assessments. This feedback was incorporated into Edition II of the evaluator 
guidebook, and the SLO team has created a rigorous and hands-on SLO training that will be launched as a part of the 
summer Academy. 


EPSS: 


The EPSS system is on-track and the team has finalized the forms and screens for the EPSS components that will be active 
starting August 2012. The team is meeting with non-RI Model districts as well as RI Model ‘non users’ (e.g. [LEA 3]) to highlight 
the EPSS features. Upcoming milestones include: 


Design validations and user-interface for the personnel data collection module (June) - COMPLETE 
Finalization of EPSS business requirements (May/June) - COMPLETE 
Planning for summer technical training and fall system administrator training (May/June) - COMPLETE 


The panel suggested several steps to encourage LEA adoption of the EPSS, including outlining in a clear communication to 


the field what MUST be done in the EPSS and the BEP requirements and standards that would be satisfied in the EPSS. 
Since then, the team discussed this at IOC and with the Data Team. Do those tasks still apply, or has the need changed? 


The team has been talking to the Innovation districts and West Warwick. How are those conversations progressing? 


IV. RISKS AND CHALLENGES: 
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The FFTPS observation tool needs to be integrated into the Single Sign-On tool. This requires matching certification IDs 
to the evaluators, and matching that information to the SSO User IDs. 

Some week 1 Academy participants had difficulty logging into the FFTPS observation calibration tool on Day 1 of the 
training. The majority of the issues were related to email SPAM filters or inability for RIDE to link the evaluator with a 
certification ID (and, therefore, they did not have SSO accounts created). The team is making adjustments to their 
communication with Academy 1 participants. 

The communication around requiring online training and tests for observation calibration (using the FFTPS) has been 
distributed through different venues; however the team continues to get pushback from districts and participants. 
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APPENDIX 3: LEA Participation 


Summer Academy Training Registration (As of July 11™, 2012) 

The goal for training registration is that at least one person per building register for teacher evaluator training and that 
at least one district representative register for building administrator training. As of July 10", the following LEAs have 
not yet met our training registration goal. 


Educator Evaluator Academy Registration Building Admin Evaluator Academy 
% of LEA Schools with at least 1 evaluator registered for At least 1 building admin evaluator 


LEA % Registered | # Remaining LEA 
LEA 1 83% 1 
LEA 2 77% 3 
LEA 3 60% 2 


LEA has not yet meet goal 


Academy registration may have been discussed as part of the action items. What outreach has been done to the districts that have 
not yet fully registered for training? 


LEAs reporting Summative Ratings (as of July 11", 2012) 

As of July Ee 10% (5 of 50) LEAs have reported their summative ratings for both teachers and building administrators. The deadline 
for LEAs to report their Summative Ratings is July a7", 2012. Note: the 50 LEAs does not reflect those that may not report data from 
this year. 


LEA Summative Ratings 
% Teachers Ineffective/Developing and Average Teacher Rating 


100% 
80% 
60% 
40% 


20% 


0% 


m% Ineffective or developing § %Effective ™m% Highly Effective 


There are only 5 districts that have reported summative ratings at this point. My suggestion would be that the panel should not 
spend too much time on this, but rather use it as an opportunity to identify what supplemental information (if any) might make this 
a rich conversation during the August 23 EdStat meeting. 
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APPENDIX 4: Evaluator Summer Academy Training Survey 


The following data is from week 1 of the summer Academies. The pre-survey baseline was gathered from evaluators the 
week prior to the training academy. The Training Data was gathered at the conclusion of each day. 


| understand the core aspects of the RI | understand the rubrics & process for 
Teacher Eval Model v2 [DAY 1] evaluating professional practice 
standards [DAY 1] 


Pre-Survey | Training Data | Post-Training Pre-Survey | Training Data | Post-Training 
Baseline Baseline 


mam Agree/Strongly Agree mums Agree/Strongly Agree 
== Net Positive (somewhat Agree, Agree, Strongly Agree) =f Net Positive (Somewhat Agree, Agree, Strongly Agree) 


The data suggest that understanding of the RI Model Edition II greatly increased after the training. Does the team have any 
observations over the impact that reading the guidebook may have had on the pre-survey data? 


| understand the rubrics & process for | understanding the process to set and 
evaluating Professional Foundations [DAY 2] approve high quality SLOs [DAY 3] 


95% 


Pre-Survey | Training Data | Post-Training 
Baseline Data 


Pre-Survey | Training Data | Post-Training 
Baseline 


mums Agree/Strongly Agree 


mums Agree/Strongly Agree 


== Net Positive (somewhat Agree, Agree, Strongly Agree) == Net Positive (Somewhat Agree, Agree, Strongly Agree) 


The day 3 content is around the Student Learning Objectives and, in the above graph, the ‘training data’ is an average of several 
nuanced question regarding understanding of how to set, define and assess evidence for, and rate SLOs in a calibrated and normed 
way. The baseline data for SLOs indicated lower levels of understanding than in other areas (which were to be expected). The data 
following the training session, though, confirms that this is still the case. 


The panel should use this data to frame a conversation around the supplemental strategies that they will implement during this 


upcoming year in order to deepen evaluator confidence and understanding in the Student Learning Objective process as a means for 
assessing educator effectiveness. 


8| Page Excellent Educators 


Quality of the Traning 
% of Academy Week 1 participants surveyed 


Day 1 (N=104) Day 2 (N=114) Day 3 (N=97) 


Academy 1 (N=149) 


mam % Rated quality of training 'high or very high’ (by Day) 
—&% Rated quality of training 'somewhat high, high or very high’ (running average of Academy) 


What was the Day 3 content, and why did less people rate the quality of the training as ‘high or very high’? 
How does this data compare to the goals that the team has set for themselves? 


Effectiveness of Facilitators in Explaining Content 
% of Academy Week 1 participants surveyed 


Day 1 (N=104) Day 2 (N=114) Day 3 (N=97) 
Academy 1 (N=149) 


mas % rated facilitator effectiveness ‘effective or highly effective’ (by Day) 
=—&% rated facilitator effectiveness 'somewhat effective, effective or highly effective’ (running average of Academy) 


Evaluator Confidence in Ability to Implementation Content 
% of Academy Week 1 participants surveyed 


94% 


Day 1 (N=104) Day 2 (N=114) Day 3 (N=97) 


Academy 1 (N=149) 


mums % Rated self-confidence to implement 'confident or strongly confident’ (by Day) 
== % Rated self-confidence to implement ‘confident or strongly confident’ (running average of Academy) 


Given that the ‘confidence’ of evaluators to implement the day 3 content is much lower than Day 1 or Day 2, what strategies are 
being implemented to supplement the week long academy training? Will they reassess this after those strategies? 
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Appendix B 


Race to the Top Collaborative Learning 
for Outcomes (CLO) 


Group 1 —- May 15, 2012 


Reform Support Network B 


Race to the Top Collaborative Learning for Outcomes (CLO) Group 1 — May 15, 2012 


LEA RTTT Project Status (All LEAs) 
Quarter 1 - Quarter 3, 2011 — 2012 School Year 


100% 


26% 22% 24% 26% 


80% . 36% __ 34% __ 
60% 
40% 
20% 


0% 


STUDY OF THE STANDARDS MODEL CURRICULUM EDUCATOR EVALUATION RECRUITMENT (School Spring) INDUCTION 


mOnTrack(Green) © Delayed (Yellow) At Risk (Red) Not Reported 


CLO Group 1 RTT Quarterly Progress Update Dashboard 
Quarter 1 through Quarter 4, 2011 — 2012 School Year 


INDUCTION 


LEA 


District A 
District B 
District C 
District D 
District E 


Summary of Accomplishments and Challenges by BEP Capacity—CLO Group 1 
Quarter 4 Race to the Top Progress Updates, 2011 — 2012 School Year 


Accomplishments 


Challenges 


Leadership Completed on-line webinar for teachers for both math and ELA. Building | Need to prioritize between different training opportunities—many 
leaders are now using these materials during faculty meetings and/or changes all in a limited time frame 
Common Planning Time to Study the Standards. Upcoming contract negotiations 
Completed evaluations Concern over the viability of maintaining an Induction Coach model 
District principals working to complete teacher evaluations of teachers after RTT funds are depleted 
District representation has attended EPSS training sessions as we 
prepare for revised model and its electronic components 
Content Participating in a consortium to write ELA CCSS aligned curriculum There are concerns this year about the weight of the SLOs as it 
Continuing work creating detailed units of study for K-12 by June 2012 relates to the rating of educators. 
Currently examining changes to the RI Model and planning Concern about clarifying changes in the ed eval model 
dissemination of information and training for staff 
In-process of surveying staff for feedback on evaluation system 
Compiled a list of improvements/clarifications to ed eval system that 
will be incorporated into a revised handbook 
The district has received positive feedback from this year's participants 
in the induction program 
Intensive work to build content expertise and curriculum in the CC 
standards across grade levels and subject areas 
Infrastructure | District evaluation committee is in place and is holding regular meetings | There is a lack of resources in terms of Professional Learning 
The district continues to recruit via School Spring, but is pro-actively facilitators that can assist in supporting teachers as they grapple 
reviewing job descriptions and pursuing early postings with new standards and curriculum in designing lessons that reflect 
District staff have applied for induction coach positions the instructional shifts that need to be made according to the CCSS. 
Short window for participating in many training opportunities 
Concerns about the amount of time needed by administrators to 
complete end of year evaluations and the evaluation system overall 
Supports Staff are being trained in new assessment systems Time—looking for creative ways to gather blocks of time for 


Completed the evaluations of all appropriate educators 

New teachers are participating in the induction program 

Regular meetings between induction coaches and school/district admin 
Completed training certified staff in the ELA CCSS and will have all 
certified staff trained in the Math CCSS by the end of the academic year 
The development of PD for CCSS implementation continues, as does the 
investigation of resources 


teachers to work together without too much disruption to the 
instructional day 

Determining next steps to support teachers in implementing CCSS 
There are challenges with CCSS roll out, ensuring that all teachers 
are comfortable rolling out the new scope and sequence in the fall 
and scheduling ongoing PD and support for teachers on CCSS roll 
out 

Loss of instruction time and availability of teachers for trainings 


Appendix C 
The EdStat Process in Action 


Reform Support Network Cc 


CLO Quarterly 
Progress 
Updates 


Completion of 


Major Tasks/ 
NAT eksike) alas 


Impact Data 
From Surveys 


KEY QUESTION: 


How well are we implementing our Induction 


Coach program for beginning teachers? 


EdStat Memo: 
Dele weliaas 


1) 50% of LEAs reported that 
they were “delayed” with 
beginning teacher induction 
aa}e)(=lantclauecldcelamel lal ays 

(o[UF-] an) goa es) 


2) The team was unable to 
finalize caseloads until October— 
well past their ideal September 
timeframe 


3) Satisfaction rates among 
beginning teachers and 
administrators were high once 
teachers were assigned toa 
coach 


EdStat Meeting: 
Panel Questions and 
Team Answers 


1) What caused the LEA 
“delayed” status? 

Answer: Administrators did not 
fully understand the program and 
thought they were off track in 
implementation. 


2) What caused the team’s delay 
in assigning caseloads? 

Answer: Statewide, retirement 
notifications were late, which, in 
turn, delayed LEA hiring 
timelines. 


3) Why do administrators rate 
the project as “delayed,” but 
express high satisfaction? 
Answer: Administrators with late 
hires did not attend orientation 
sessions, and therefore know less 
about the program (and rated it 
as “off track”). 


EdStat Meeting: 
Sample Action Items 


e Determine potential 
opportunities for “data sharing” 
with retirement board 


e Identify “new to program” 
administrators and work one- 
on-one with them so that they 
understand the Induction Coach 


| model and expectations 


e Leadership team should 
identify funding implications of 
a higher-than-expected number 
of beginning teachers 


e Identify districts early on with | 
a high number of anticipated 
beginning teachers 


